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ABOUT THIS CASE STUDY 


Achieving the Dream (ATD) is one of 12 higher education and digital learning organizations that 
make up the Every Learner Everywhere (Every Learner) Network, whose mission is to help higher 
education institutions improve and ensure more equitable student outcomes through advances 

in digital learning, particularly among poverty-impacted, racially minoritized, and first-generation 
students. Every Learner partners are addressing high failure rates in foundational courses through the 
provision of scalable, high-quality support to colleges and universities seeking to implement adaptive 
courseware on their campuses. As part of its ongoing effort to helo community colleges develop 
effective teaching and learning practices, ATD is working with seven community colleges in Florida, 
Ohio, and Texas on this initiative, providing coaching and direct support to the colleges, fostering 
collaboration within and among the participating institutions, and serving as a liaison to the Every 
Learner network. 


The following case study is part of a series of studies conducted by ATD examining how adaptive 
courseware is implemented at those institutions as well as how courseware is used in particular 
disciplines to better serve students. Case studies are based on a series of interviews with college 
leaders, faculty, instructional designers, developers, technology specialists and students who were 
enrolled in classes using the Courseware. 
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OVERVIEW 


Gateway science courses for both STEM majors and 
those completing general degree requirements 
require students to master complex concepts. Always 
challenging, these courses have been made more 
difficult for many students by the transition to online 
learning during the pandemic. Colleges participating 
in the Every Learner Network found that adaptive 
courseware engaged students and supported their 


learning. Among their findings: 


¢ The approach to guided review 
and assignments in adaptive 
courseware aligns with the 
discipline’s approach to 
conceptual and applied learning. 


¢ Participating faculty reported 
that adaptive courseware 
increased engagement and, at 
times, academic success, among 
students who actively used it. 


¢ Asin other disciplines, faculty 
and students both stressed the 
importance of acclimating 
students to adaptive work, 
monitoring progress and 
intervening when individual 
students struggle, and ensuring that 
adaptive assignments align with 
course content and objectives. 


e Students reported that 
courseware could be more 
challenging and time-consuming 


than traditional assignments 
in science courses but 
acknowledged that adaptive 
activities heloed them learn 
complex concepts. 


B SUPPORTING INSTITUTIONAL REFORM 


The Every Learner initiative supports broader efforts to 
foster student learning with evidence-based practices, 
including supporting students with foundational skills 
for the gateway courses required for both STEM and 
general degree programs. “We need to teach students 
how to use the information they learn to solve a wide 
variety of problems,” says Amarillo College chemistry 
professor Jennifer Rabson. It also reflects the contexts 
in which broader institutional reform is taking place 

at community colleges throughout the ATD Network, 
including building a culture of excellence in teaching 
and learning and leveraging data and technology 

to support student success and equitable student 
outcomes. To learn more, see p. 7. 
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Three of the seven community colleges in the Every Learner Network piloted 
adaptive courseware in science courses, focused on gateway biology, 
chemistry and physics courses. 


INSTITUTION ore) US) COURSEWARE 


Indian River State College 


Fort Pierce, Florida Introduction to Chemistry HS oreo ileal nulc 


Sparrow) 
16,686 students 
Introduction to Biological 
Chemistry 
Anatomy and Physiology Mastering (Pearson) 
Everyday Chemistr k 
Cuyahoga Community College . “ ‘ ; eon 
Introduction to Inorganic MyLab (Pearson) 
Cleveland, OH Ghonneh 
DU asia Anns y LearnSmart/Connect (McGraw- 
Sec General Chemistry | Hill Education) 
General Chemistry II ALEKS (McGraw-Hill Education) 


College Physics | 
General Physics | 


Introductory Chemistry | 
Amarillo College 
Gen Organic & Biological Chem 
Amarillo, TX or Knewton Alta (Wiley) 
Principles of Chemistry | 
9,739 students 


Principles of Chemistry Il 


For more detail on the pilot efforts at each college, see case studies for each institution. 
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INTRODUCTION 


Jordan Kadas’s experience in her introductory chemistry 
course at Cuyahoga Community College wasn't that 
different from many of her peers in community colleges 
across the country. “| was mixed in with people who 
hadn't gone to school for 10 years, while others had basic 
chemistry knowledge and there were some high school 
students in my class,” says Kadas, who graduated from 
the Ohio college with an associate degree in 2019 and is 
now pursuing a nursing degree at a four-year institution. 
“There were really different knowledge levels, and it 
made each of our (Course) experiences very different.” 


Those varied experiences are Adaptive software, says Amarillo 
also what prompted chemistry College chemistry professor Jennifer 
professor Dr. Anne Distler to begin Rabson, “is especially good for 
experimenting with different science courses because we need 
technologies to improve her online to teach students how to use the 
courses. “There’s a lot of math and information they learn to solve a 
science phobia,” says Distler, a wide variety of problems — to apply 


faculty member at the Ohio college, it outside the box.” 
known locally as Tri-C. “There’s a lot of 
concern as students start launching 
into a class they‘re not comfortable 
with. | realized there needed to 

be a better way to reach students 
engaging on online platforms.” 


At Tri-C and other community 
college campuses participating in 
the Every Learner Initiative, faculty 
have explored ways in which 
adaptive courseware can help 
students in challenging STEM courses 
succeed. What they and students 
like Kadas have found is that while 
the technology can be challenging, 
and at times frustrating, it can be 
effective in helping students learn 
complex concepts. 
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‘GO ON A LITTLE JOURNEY’ 


Dr. Heather Michaels likens chemistry lessons to a 
“journey” — and adaptive courseware to a productive 
way of retracing steps during that journey in the 


service of mastery. 


“You're learning the concepts, 
bringing in some guided 

reading, and there are places 
where students are asked if they 
remember the concepts,” says 
Michaels, a master instructor of 
physical sciences at Indian River 
State College in Florida. “If they 
say no, they have to go on alittle 
journey to relearn it.” 


Given the emphasis on translating 
complex concepts into real-world 
practice, science is an ideal 
discipline for adaptive courseware, 


faculty members at institutions 
participating in the Every Learner 
initiative say. Rabson focuses on 
helping students “walk through 
the problem-solving process when 
they learn this material,” she says, 
with adaptive work reinforcing this 
overarching skill through guided 
review and practice. 


“So much of science is just learning 
how to think and how to approach 
problems,” Rabson says. “| feel 

like when | was teaching more 
traditionally, | would try to help 
students by giving them examples 
of everything, and then they would 
echo and repeat my examples 

on the test, but they might not 
understand what they did.” 


Distler agrees. “The questions | like to 
assign are very conceptual, sorting, 
or visual, and very similar to what I'd 
like to do in a class or lab,” she says. 


“In science, you want to visualize 
the way putting concepts together 
works,” adds Tri-C physics professor 
Jonathan Williams. “Based on 
what product you use, the types of 
animations are better than what | 
could put on the board.” 


Like other community 
colleges participating in the 
Every Learner grant which 

are part of the ATD Network, 
the three institutions profiled 
in this case study have 
committed to engaging in 
bold, holistic, and sustainable 
institutional change across 
multiple institutional areas 
and priorities. Their efforts 

to implement adaptive 
courseware to support 
students in the gateway 
science courses required for both STEM and general 
degree programs reflect the importance of several 
key cornerstones of institutional change, including 
leveraging data and technology to support student 
success and equitable student outcomes and 
building a culture of excellence in teaching and 
learning. 


AID’s Institutional Capacity Framework and 
Institutional Capacity Assessment Tool (ICAT) outlines 
seven essential institutional capacities required to 
create a student-focused culture that promotes 
student success. One focuses specifically on teaching 
and learning and the commitment to engaging 
full-time and adjunct faculty in examinations of 
pedagogy, meaningful professional development, 
and a central role for faculty as change agents within 
the institution. Building capacity in this area is crucial 
because, as ATD President Dr. Karen A. Stout recently 
asserted, “focusing on teaching and learning is still 
not central to the field’s overall theory of change. We 
still have much more to do to build a deep focus on 
pedagogy and to support our colleges in building a 
culture of teaching and learning excellence.” 


To foster this culture of teaching and learning 
excellence, AID’s Teaching & Learning Toolkit: A 
Research-Based Guide to Building a Culture of 
Teaching & Learning Excellence is centered on 
four cornerstones of excellence that provide a 
forward-looking vision that campuses can use to 
inform their work. 


fe SWI ING ONPAlID Ss CORNERSTONES OF EX CELMRENCE 


Initiatives such as Every 
Learner provide important 
resources and supports to 
community colleges and 

the time, space, support, 

and resources to explore 
innovative pedagogical 
approaches to improving 
student learning and 
outcomes. They also offer 
sustained opportunities to 
build on these cornerstones 
of excellence. At these three 
institutions, the grant provided 
science faculty with the time and resources needed 
to evaluate, implement, and modify their use of 
adaptive courseware to best serve students. 


Efforts to provide needed support and basic skills 
building for students exemplifies institutional efforts to 
empower faculty to consider, adapt, test, and refine 
new approaches to fit their campus context and the 
needs of their students. In science courses, this work 
often focused on identifying ways to strengthen math 
skills, with Courseware playing a key role in getting 
students “prepared for the material that was going to 
come,” says Tri-C chemistry professor Dr. Anne Distler. 
Adaptive course work also helped students master 
the highly conceptual content taught in science 
courses and “apply it outside the box,” says Amarillo 
College chemistry professor Jennifer Rabson. 


At some institutions, collaborative partnerships 
included adaptive courseware in efforts fo support 
student success. For example, Tri-C’s first-day 
access initiatives provided automatic access to 
courseware when students registered. “Taking one 
barrier out of that has made a huge difference for 
students,” Distler says. 


Faculty also worked to use data from adaptive 
courseware to monitor student progress and 
refocus supports and the content of their courses. 
“It really heloed me see where they were,” says Dr. 
Heather Michaels, a master instructor of physical 
sciences at Indian River State College. 
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TRIAL AND ERROR 
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you can get away with it. With adaptive courseware, there’s no way to get 
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— Sinem Balta Beylergil, Tri-C student 


Participating faculty members say that adaptive 
courseware is particularly effective in helping 
students review concepts and prepare for class 
sessions. Nancy Doherty has students review 
concepts before twice-weekly synchronous 
meetings in her chemistry courses at Tri-C. “When 
you're face to face, you want students to have 
some familiarity with the material,” she says. 


Williams agrees, noting the time constraints of 


the compressed synchronous classes he teaches. 


“| don’t have time to go through every single 
concept, so | would have expected students to 
go through the adaptive assignments,” he says. 
“| asked students to read — they don’t always 
read. But doing the dynamic study module is a 
nice way to go through the entire content before 
coming into class.” 


Many faculty members also use Courseware to 
provide guided homework assignments. Since 
science questions tend to be complex and 
require students to work through multiple steps, 
adaptive courseware can provide support 

for “when I’m not able to be there to answer 
questions for them,” says Distler. 


“| like how it would give you additional questions 
on your weak areas and immediate feedback,” 
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adds Williams. “If we didn’t have these tools, a 
student would submit an assignment on paper, 
and it would be a week before feedback is 
delivered to them. The adaptive piece is to have 
that immediate feedback without waiting for the 
professor fo walk you through something.” 
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courses, and the differences 
between how the professors used 
them was huge. What | got out of 
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As in other disciplines, science adaptive 
courseware allowed faculty to track student 
progress and identify those who needed 
additional support. “Students who were doing 
extra work generally accepted they needed the 
extra time and worked through it,” Rabson says. 
“The biggest problems are students who never 
warmed up to (the courseware) or decided they 
weren't going to do homework. They may have 
never gotten fully committed to the class.” 


THE IMPACT 


&& Courseware brings some system to the learning. 
If there’s nothing to follow and it’s just a textbook 
and exams to be taken down the line, | feel a little 
lost. It gives me some sort of schedule and more 


frequent deadlines. Rewards, reminders — that kind of 
feedback is really helpful. 99 


sinem Balta Beylergil, Tri-C student 


Across participating institutions, 
faculty members said that students 
are more engaged and successful 
when they work with adaptive 
courseware. At Amarillo College, 
students benefitted from more 
hands-on lab activities in class 
because “faculty saw what they 
needed to spend time on” through 
data generated by the courseware 
and could focus their lectures, says 
instructional designer Dr. Lori Petty. 
At Tri-C, faculty said that adaptive 
courseware helped students stay 
on track. “In particular, students 
completed more assignments in 
adaptive courseware", says Williams. 


“lsee a huge correlation between 
the students who do the electronic 
homework and those who don‘t and 
how they do in class,” agrees Doherty. 


Tri-C biology assistant professor Dr. 
Prabhat Sharma agrees. “Adaptive 
(courseware) gave students three 
chances,” he says. “In their first 
attempt, they were not getting 
good (scores), but they knew what 
they had to work on and had more 


chances to work on weaknesses and 
deficiencies, and grades went up.” 


Students generally found adaptive 
courseware challenging but 
ultimately beneficial. For example, 
nondegree student Sinem Balta 
Beylergil took different chemistry 
courses with and without adaptive 
courseware at Tri-C. “| felt there was 
too much work and | ended up 
spending more time compared to the 
other course without Courseware,” 
she says. Even so, “a month later, you 
remembered what you had trouble 
with... If makes you learn — that’s the 
pain associated with it.” 


At Tri-C, including science courses 
with adaptive courseware in 

first-day access initiatives, which 
automatically include access to 
courseware when students register for 
class, has also boosted engagement. 
“| had a hard time getting students 
to get the books,” says Distler. “In 

the last two semesters, | have had 
amdajority of students log in and 
complete assignments in the first two 
days, which |’d never seen before.” 
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ADAPTIVE COURSEWARE IN PRACTICE: SUCCESSES AND CHALLENGES 


What Worked Well: 


Remediation: Faculty found adaptive 
assignments based on prerequisite knowledge 
particularly helpful. “| found a lot of students 

were not ready for the course. They needed a 

lot of math intervention,” says Tri-C chemistry 
professor Dr. Anne Distler. She found that adaptive 
questions from an introductory chemistry class 

for non-science majors helped prepare students 
in gateway classes for chemistry majors who 
needed intervention on prerequisite concepts. The 
assignments “got them situated and prepared for 
the material that was going to come,” she says. 
Amarillo College chemistry professor Jennifer 
Rabson says the variety of questions and topics 
within adaptive courseware is also a benefit. 

“It is advantageous to have a wider variety of 
questions to force students to start thinking through 
what theyre doing and how they would solve 
different types of problems,” she says. “If a student 
is having trouble with a certain type of question, 
the more it will try fo lead them to the things that 
will helo them to figure out that concept.” 


Adaptive tools that support complex science 
problems. Faculty members said adaptive 
products include built-in tools to enter and 
structure complex math equations and for 
measuring and drawing that allow for more 
sophisticated homework problems. 


Detailed feedback and guidance. Some — but 
not all — adaptive products provided feedback 
and links to reinforce key concepts through each 
part of complex, multistep questions, faculty 
members said. “The (questions) that step them 
through and grade each step — those are 
phenomenal,” Tri-C chemistry professor Nancy 
Doherty says. “If they’re drawing an organic 
structure and have the wrong number of bonds 
to a carbon, it'll guide them. In class, I'd draw it 
very similarly.” 


Even in graded assignments, courseware could 
generate a similar practice question and walk 
students through the steps fo solve if so they 
“don’t waste the attempt,” Distler says. “I have 


he ACHIEVING THE DREAM ¢ EVERY LEARNER EVERYWHERE 


had students asked for nongraded assignments 
that have that practice, and I’ve done that to 
helo them.” 


Guided review. Faculty found adaptive 
products provided a comprehensive review 

of course objectives. “| like it because it asks 
questions on all topics, but if you know it, it gives 
you fewer questions,” Doherty says. “It does some 
adaptive work for the students, but it also gives 
them the chance to look aft all the materials.” 


Enabling targeted support for students. Faculty 
said they monitor student progress in courseware 
to determine where to intervene. “If someone’s 
doing it on the first attempt and someone’s 
taking five or six times, it helps me identify the 
ones | need to reach out to,” Distler says. “It helps 
me get to the heart of what’s going on with the 
individual student that | couldn’t if | didn’t have 
that data.” 


Customizability. Rabson says the ability to 
customize adaptive courseware is critical. “You 
can pick and choose assignments and then 
objectives depending on exactly what you 
want your students to learn, or if you know your 
students have limited time, what you want them 
To focus on,” she says. 


Cost and access. At around $40 per semester, 
adaptive products are significantly cheaper than 
used textbooks and homework packets, which 
begin at $125 or more. At Amarillo College, the 
goal is “trying not to add to students’ financial 
burden,” says Rabson. “It’s greatly improved 
things.” In similar fashion, Tri-C’s first-day access 
program helped improve access to materials. 
“Students always had the ability to sign up for a 
free trial, but taking one barrier out of that has 
made a huge difference for students,” Distler says. 


Ongoing Challenges: 


Technology overload. Students in science 
courses often are provided with a wide range 
of online materials and resources, including lab 
materials, textbooks, and other digital tools, 


and adaptive courseware can seem like yet 
another disconnected option. “There’s always 

a new tool for them,” says Doherty. “When I’m 
teaching face to face, they wonder why they 
have to do online homework. | try to make them 
understand they'd be doing it on paper if they 
weren't doing if online.” 


Managing workload and seftings. As in other 
disciplines, faculty said students offen found 

it difficult to progress in some Courseware 
assignments when they struggled with concepts. 
Doherty found “a lot of hard questions and not 
enough easy ones” in courseware for her entry- 
level introductory courses. While faculty members 
may not always be aware of how to adjust 
assignments, others said that some products 
don’t allow them to override key adaptive 
settings. “| had a student who had a nervous 
breakdown in my office because she got into a 
loop and there was no way for me to get her out 
because | couldn’t circumvent the structure of 
the tool,” Distler says. “There was no control for 
me to say ‘we're going to skip that question’.” 


Tri-C biology assistant professor Dr. Prabhat 
Sharma adds that it’s not always clear what 
problems students are receiving in some 
adaptive products. “| don’t know if they’re 
getting the right level of questions,” he says. “| 
should be able to see what they're getting.” 


At the same time, Doherty says some adaptive 
homework was “gameable,” meaning students 
could progress without demonstrating mastery 
of the concepts. “You can push buttons offen 
enough to get the right answers without doing 
any thinking,” she says. 


Onboarding. Students said that learning how 

to use the software could be difficult. “| was 
overwhelmed,” says Tri-C student Jordan Kadas. 
“| figured it out, but! would have loved a tutorial 
to begin with.” Another Tri-C student, Sinem Balta 
Beylergil, agrees. “| work with computers and 
online programming, but it just didn't feel intuitive 
at all,” she says. 


Faculty members have responded with 
introductory videos and assignments to familiarize 
students with Courseware, but some say that 


onboarding has become more challenging since 
the rapid pivot to online courses in Spring 2020. 
“Students would be more comfortable if we had 
some type of in-person contact,” says Tri-C physics 
professor Jonathan Williams. “Right now, students 
are having problems familiarizing themselves with 
the software. If we were meeting once | week, | 
would be able to (better help them.” 


Lack of feedback through multiple steps 

of complex questions. While some adaptive 
products earned high marks for providing 
feedback and guidance through complex, multi- 
step problems, faculty members said that other 
courseware was less effective at providing step- 
throughs. “It gives you a right or wrong answer, 
but it doesn’t walk you through the calculations,” 
says Williams. “One thing students said they'd like 
to see better is not just being told the answer is 
wrong and what it is, but walking them through it.” 


Dr. Heather Michaels, a master instructor of 
physical sciences at Indian River State College, 
agrees. “Other products were more like homework 
systems where it was like ‘here’s a question, 
answer the question,’” she says. “It didn’t feel as 
much as a teaching tool as a testing tool.” 


Other faculty members said that different 
courseware provided step-through support for 
some — but not all — adaptive questions, and 
some said they preferred less adaptive products 
that provided more comprehensive feedback 
and support for students. 


Learning curve for faculty members. More than 
in other disciplines, science faculty said it was 
difficult to learn how to use adaptive courseware 
— and particularly how to craft assignments at 
the appropriate level of difficulty for students. “It’s 
a really steep learning curve for faculty and so 
different from other courseware,” Doherty says. 


Siloed questions. Faculty which experimented 
with multiple Courses or products found that 
questions are “often siloed by textbook,” Doherty 
says. “They have a huge database of questions 
you could use, but you can only access the ones 
(associated with the textbook) you use. It seems 
like a waste.” 
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LESSONS LEARNED 


Keys to the successful implementation of adaptive 
courseware in science courses across Every Learner sites: 


¢ Identifying specific adaptive 
components to meet course 
needs. Given that many 
participating faculty members tried 
more than one adaptive product, 
they stressed the importance of 
selecting products that align with 
their classes. “You try to choose a 
textbook that best fits your learning 
style and meets the objective of 
the class, and you choose software 
that complements your teaching — 
the ones that have the same types 
of questions, the same graphs and 
figures that complement your in- 
person teaching,” says Tri-C physics 
professor Jonathan Williams. 


That goal is made easier by the 
fact that faculty can select which 
components of courseware they 
use. “A lot of the publishers have 
an adaptive piece, a homework 
piece, and other pieces you can 
use,” says Tri-C chemistry professor 
Nancy Doherty. 


¢ Aligning courseware with 
course objectives. As in other 
disciplines, students at times 
reported that adaptive work 
didn‘t match up with course 
lectures or tests. “I felt like | was 
taking two different classes — the 
one the professor was teaching, 
and what | was learning from (the 
courseware),” says Tri-C student 
Jordan Kadas. 


To address these concerns, Amarillo 
College chemistry professor 
Jennifer Rabson deliberately maps 
out adaptive assignments and 
readings before each semester 
and details them in the course 
syllabus and on Blackboard so 
students have a clear roadmap 

to follow. “It works well if you spell 

it out exactly and make sure the 
content matches,” she says. “You 
just have to have a plan for what 
you can realistically accomplish 
with everything else students have 
going on in their lives.” 


The importance of onboarding. 
“It’s critical to have a strategy that 
acknowledges that students are 
going to have a learning curve 
and build that into the course 
during the first couple of weeks,” 
Distler says. For example, both 
Tri-C chemistry professor Dr. Anne 
Distler and Doherty structure 
activities around familiarizing 
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prepared me for my tests because it was so 
beyond what | needed to know. 99 
— Jordan Kadas, Tri-C student 
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students with the courseware 

in the first few class sessions, 
including a syllabus quiz and 
introductory assignments focused 
on “concepts they already know 
so they can get used to the tools,” 
Doherty says. Throughout the 
course, Doherty solves a problem 
in adaptive courseware in each 
class, “showing them what it’s like 
for me to do it online.” 


Tracking student progress. 
Faculty members stressed the 
importance of monitoring student 
progress and referring students who 
are struggling to tutoring or one-on- 
one support. “There’s a tendency to 
believe adaptive means you don’t 
have to do anything in the course,” 
Distler says. “It’s not on autopilot — 
you need to interact and intervene 
with your students and use the 
data tools to give the student the 
best experience." 


Faculty said adaptive products 
offered differing levels of insight 
into student progress. Dr. Heather 
Michaels, a master instructor of 
physical sciences at Indian River 
State College, began taking 
students into the computer 

lab to monitor their progress in 
courseware and identify which 
ones were having trouble using 
it. “It really helped me see where 
they were,” she says. 


Consistency. To avoid the often- 
heard challenge of students 
allowing adaptive assignments 
to pile up over weeks or months, 
Doherty stresses the importance 


— 


of building a consistent schedule 
of assignments in which students 
are working in the Courseware 
every few days. “lam lenient with 
extensions, but that forces students 
to think they need to work on this 
every few days or they won‘t be 
able to keep up,” she says. 


Michaels agrees. “| had students 
who would not soend much time 
with the software — it was really 
disheartening,” she says. “Weekly 
objectives helped.” 


As a student who took 
consecutive chemistry Courses, 
Kadas suggested an additional 
approach to consistency: using 
common courseware across 
course sequences. “I think it would 
be really beneficial and allow 
students to get ahead,” she says. 
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CONCLUSION 


More than their peers in other disciplines, participating 
faculty members said the transition to online Courses 

in Spring 2020 was challenging for students in science 
classes, which often include hands-on and small- 
group activities that help support students and keep 
them engaged. “In the current environment, students 
aren't excited about learning online but doing it out 

of necessity,” Distler says. Adaptive courseware has 
provided important support for these students, she adds: 
“Since I’m not seeing them face to face, it’s guiding 
them in a way | can’t do one on one” during online 
courses. “There are still a lot of students struggling in this 
current environment, but | think they're getting there.” 


While Kadas found the level of work within adaptive 
courseware too difficult for her introductory chemistry 
class, she said it was “invaluable” during a subsequent 
higher-level course. “It heloed me understand some 
bigger concepts,” she says. “(Adaptive) is definitely a 
challenge, but! would absolutely take any other course 
with it. | got so much out of it.” 
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